A portion of the disclosure of this patent document contains material which is subject to copyright protection. The owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.
The present invention relates generally to the field of computer graphics and specifically to the field of creating a database with improved performance and resolution for large areas rendered in real time. The present invention presents a method and apparatus for creating a database that enhances the resolution of a scene and renders more efficiently in real time.
Typical graphics rendering systems utilize a geotypical texture database, wherein geotypical textures are stored in memory and utilized to fill in textures for every occurrence of a particular texture, for example a cornfield. The geotypical cornfield texture is held in memory and used repeatedly on every polygon that represents a small section of the cornfield. Typically, there are many corn field texture polygons requiring the geotypical cornfield texture to be used thousands of times in a scene containing cornfields. Thus, such a geotypical database would use a large number of polygons to represent the surface of the earth.
Real time rendering requires that each polygon undergo a mathematical transformation, a matrix multiplication within each display frame to map each polygon from world coordinates to display coordinates. Typical graphics rendering systems have limited processing power to render a large number of polygons in real time, thus the presence of the polygons slows down system performance during real time rendering. Thus, there is a need for a method and apparatus that reduces the number of polygons that must be processed in real time rendering.
Current rendering systems can display large geospecific images, however, resolution of those graphics rendering systems is limited to the resolution of the source imagery. Thus, there is a need for a method and apparatus that enables enhancement of rendered imagery beyond the resolution of the source imagery.
There is also a shortage of high-resolution imagery data coverage worldwide. In many cases only low-resolution imagery is available and in some cases no imagery is available. Thus, there is a need for a method and apparatus that enables the creation of real time scenery in areas where there is no source imagery available.
The present invention provides a method and apparatus for building a database off-line that can be used to efficiently render a scene in real time. The present invention enables creation of a database comprised of high-resolution models incorporated into lower-resolution source imagery. The present invention renders and captures an orthographic projection of a scene and stores it as a pixel representation of the rendered imagery and the models. The pixel representation is stored in a database for rendering in real time. The resulting database can be rendered more efficiently in real time than databases built using previously known techniques.
The database provides improved resolution and improved rendering performance. Scene resolution is enhanced by substituting images of high-resolution geotypical textures and models in place of lower-resolution source imagery. The present invention builds a database generated from an orthographic rendering of a enhanced scene wherein images of high-resolution models (SPLOT models) have been inserted into lower-resolution source imagery. The rendered representation of the scene is stored in a database for rendering in real time.
The present invention builds a database containing high-resolution pixel representations and lower-resolution polygonal representations, thereby enabling switching between the pixel representation for two-dimensional rendering and the polygonal representation of a feature for three dimensional rendering in a scene. The present invention generates a database that enables efficient rendering in real time.
The present invention removes excess polygons from culture to speed up the real time rendering process. A visual database generally includes polygons that represent the undulating surface of the terrain, plus additional polygons that represent cultural features on the terrain. The present invention reduces the polygons that represent cultural features by replacing them with textures derived from said polygons. For example, a database may contain polygons with a geotypical cornfield texture, said polygons coplanar with the polygons representing the terrain surface; and other polygons with a geotypical runway texture, said polygons also coplanar with the terrain surface; and other polygons that represent paint markings on said runway, coplanar with the runway polygons; and polygons used to represent three-dimensional features (for example, buildings) on the terrain. All such cultural polygons may be replaced by a single large texture applied to the terrain surface polygons, said texture derived from the polygons representing the cornfields, the runways with its surface markings, and the buildings. Said texture may be considered as pseudo-geospecific. inasmuch as it appears very similar to high-resolution imagery, were such imagery available.
The present invention provides a method and apparatus that enables construction of a database having fewer polygons to enable higher performance rendering in real time. The present invention also enables creation of synthetic high-resolution imagery where no source imagery is available. The present invention also provides an apparatus and method for enhancing the resolution of a representation of source imagery.
The present invention renders a polygonal representation of a scene (geometry and texture) containing high-resolution models, as an orthographic projection and then captures a pixel representation of an orthographic projection of the scene. The orthographic pixel textures are combined into a composite texture. The resulting composite texture is then draped over the surface contour. The composite texture can be rendered rapidly because the bulk of the cultural features have been transformed from a polygonal representation to a texture representation. The initial underlying polygons, representing the culture which slow down real time rendering, are no longer a part of the scene. The present invention also removes two-dimensional coplanar polygons. Thus, the present invention enables rendering a scene with a fraction of the polygons utilized in typical graphical rendering systems. The present invention provides essentially the same scene content with an order of magnitude reduction in the number of polygons, an increase in resolution and additional coverage in areas where source imagery is unavailable.
Resolution is effectively increased in selected target areas. For example, in a five-meter resolution source image, the present invention enables an operator to mark, locate and digitize features such as roads buildings and fields that appear in the image. For example, a road in a five-meter resolution image may be visible but not as sharp as desired. The present invention provides a method and apparatus for sharpening the presentation of the road in the image. The present invention provides a method and apparatus for creating models comprised of polygons and high-resolution textures to replace or create the features in an image, thereby increasing the overall imagery resolution.
The present invention enables replacement of original features with sharper, higher-resolution models. The same process can be used to create and add features such as parking lots, buildings and fields that may or may not appear in the source imagery. The resulting intermediate image comprises a combination of the five-meter resolution source imagery and the higher-resolution models. The combination yields an image with higher resolution than the original five-meter source imagery.
The present invention renders an orthographic projection of the scene including the models at higher resolution than the source imagery. For example, a one-meter resolution orthographic projection rendering is performed for a five-meter resolution source imagery scene containing sub-meter resolution model textures. Thus, the one-meter orthographic projection effectively under samples the sub-meter resolution model texture and over samples the five-meter texture surrounding the model. Thus, the orthographic projection provides higher resolution in the target area comprising the models and is sharper than the original image in the target area. Moreover, the higher-resolution imagery correlates well with the original imagery so that the enhanced image looks like the scene in the original image. The resulting synthetic imagery also correlates well with the high-resolution polygon scene.
The present invention enables an operator to select features in the source imagery for digitization and replacement with high-resolution models or he may chose to add high-resolution models for features that do not appear in the source imagery. The orthographic rendering is preferably performed by Vega, which is available from Paradigm Simulation, Inc., Dallas, Tex. and runs on the Onyx2 computer system shown in FIG. 1. There are a plurality of commercially available graphics rendering tools and hardware platforms that can be used to generate an orthographic projection of a scene.
Feature addition is useful when source imagery is lacking. Five-meter satellite imagery is scarce and when available may be classified. Moreover, there are gaps in coverage where no imagery is available. Satellite imagery also presents cloud coverage problems when clouds obscure portions of the imagery. Additionally, satellite imagery is often taken at different times of year so adjacent shots do not match and do not create a coherent mosaic, for example, summer in one section and snow covered in winter in another section so adjacent sections do not match. Thus, satellite imagery, even if available, is problematic. The present invention enables scene generation without any source imagery. The present invention enables generation of a scene from maps of surface contours and features without actual source imagery.
Correlation of source imagery can be problematic. Often database imagery is created from different sources. Oftentimes feature location data and source imagery does not correlate well. The present invention creates a database that generates imagery that does correlate well because it generates enhanced-resolution features by referencing source imagery pixels.
The present invention enables an operator or an artificially intelligent feature recognition process to recognize and digitize features from various sources, including topographical maps. The present invention enables input of features from such sources. For example, topographic maps may illustrate trails, which divide small areas into different types of fields. The present invention enables an operator to use commercially available tools to digitize these fields and arbitrarily assign geotypical textures to each field. The present invention enables an operator to locate and draw or digitize features such as primary, secondary and tertiary roads. High-resolution models are substituted for selected features so that primary roads are replaced with a geotypical model of a superhighway, secondary roads are replaced with a two lane highway model, and tertiary roads are replaced with a dirt road model.
The present invention enables creation of a geospecific database based on feature location data from a map of features, without using source imagery. Such a database enables generation of a real time database, which for example, can be used in a flight simulator to train a pilot to fly over and become familiar with a geographic area for which source imagery is not available. The present invention enables creation of imagery scene data that correlates well with actual source imagery. The synthesized scene database can be created without source imagery and looks realistic. A pilot trained on such a database would recognize the actual physical terrain, even though the database was created without source imagery.
Orthographic projection is well known graphic rendering technique for graphic rendering tools such as Vega. The present invention creates a composite texture database by systematically stepping over the orthographic projection of a scene and capturing a pixel representation, section by section. The present invention captures a pixel representation of each section of the orthographic image and places them altogether in one large composite texture pixel representation. The composite texture is mapped onto the polygonal terrain contour for the scene and placed in the database for real time rendering.
A preferred embodiment utilizes a Silicon Graphics, Inc. Performer-compatible clip map data structure for storing the levels of detail used in rendering an image in real time. The clip map contains all the resolution sets for an image. For example, for a twenty-five-meter resolution source image, the clip map would contain the twenty-five-meter resolution image, and reduced-resolution data sets at fifty meters, one hundred meters, two hundred meters and so on, at progressively lower resolutions. The present invention propagates the synthetic imagery throughout the lower levels of detail stored in the image clip map.
The present invention, for example, builds a one-meter resolution image in the clip map from high-resolution geotypical textures and then down samples that image to create lower-resolution images at two meters, four meters, eight meters, etc. The present invention continues generation of reduced-resolution data sets until it has generated and placed the geotypical texture in every resolution level of the clip map. The present invention replaces source imagery with enhanced-resolution imagery that includes higher-resolution models for features of interest. The enhanced feature model is propagated into every level of the clip map for the scene.
Thus, an original twenty-five-meter source imagery resolution data set contains a one-meter model of a road rather than the original road that was captured at twenty-five-meter resolution. Thus, the road now appears sharper than it did in the original source imagery. As an observer utilizing the database moves closer in on a displayed scene and approaches the enhanced feature, it appears at higher and higher resolution, and there is a smooth transition through all the resolution data sets presented in the clip map. For example, the one-hundred-meter imagery blends into the fifty-meter imagery, which blends into the twenty-five-meter imagery, because all resolution sets are derived from the highest-resolution image data.
The present invention enables replacement of a single feature or an entire scene with a high-resolution model embedded in source imagery. The present invention utilizes Multigen 11 to build geometric models that are then embedded into source imagery to sharpen, replace or create features for inclusion in a real time database. The present invention removes any polygons that are coplanar or insignificant to simplify and reduce the number of polygons used to represent the geometric model.
For example, to build a scene of the tabletop with a sheet of paper, cups, and an ice bucket, the present invention generates a pixel representation of the tabletop with all the objects on it. Individual polygons typically carry the image texture for the tabletop and each of the objects on the tabletop. The present invention renders the typical textures polygons for the objects laying on the table to create a pixel representation or picture of the tabletop texture with everything on it. The repeating geotypical table texture is replaced with a larger geospecific texture for the entire tabletop.
Viewing the table top scene at a distance, there is no three-dimensional perspective of the objects on the table top, thus the present invention does not render the objects on the table top with all the polygons required for three-dimensional presentation of the objects. The present invention renders a two-dimensional pixel representation of the tabletop that includes the objects on the tabletop. When the observer perspective closes in on a feature, the present invention switches from the two-dimensional pixel representation to a polygonal three-dimensional representation of the feature so that the rendered feature manifests three-dimensional relief.
Switching from a two-dimensional pixel representation of an object to a three-dimensional polygonal representation also helps to prevent loss of texture sharpness that occurs at close range when the relative size of the texels representing a feature become larger than a screen pixel. In order to retain sharpness in the image the present invention switches to the three-dimensional polygonal representation of the feature comprising a geotypical textured model polygonal representation of the feature. The resulting database enables creation of an image that renders efficiently in real time and correlates well with original source objects.
Prior to the present invention, the typical graphics rendering apparatus or method encountered performance bottlenecks due to excessive polygon usage in texture representations. Thus, to reduce the number of polygons needed to render an image in real time, the prior graphic rendering method and apparatus had to settle for coarser representations of features using less polygons or fewer features. Typically, prior methods forced a reduction in geometric complexity to achieve rendering speed. The reduction in polygons also reduced resolution and information content in a polygonal representation of an image. For example, a polygonal representation of a curved road using fewer polygons had less resolution and would make the road appear choppy when compared to a polygonal representation of the same road rendered using more polygons. This trade off between performance and image complexity has been diminished by the present invention. The present invention instead improves both performance and image complexity while increasing the resolution of a scene rendered using fewer polygons than the previous typical known methods.
In one aspect of the present invention a method is presented for building a database for real time rendering of a scene comprising the steps of substituting a high-resolution polygonal feature model for a feature in scene source imagery: systematically rendering said scene containing said high-resolution model:
capturing a pixel representation of said rendered scene containing said high-resolution model; and
storing said pixel representation of said rendered scene containing said high-resolution model in a database for real time rendering of said scene. In another aspect of the present invention a method is presented further comprising the step of systematically rendering an orthographic projection of said scene containing high resolution models. In another aspect of the present invention a method is presented for building a database for real time rendering of a scene comprising a polygonal representation having coplanar polygons and feature identification codes for polygons comprising the steps of substituting a high-resolution polygonal feature model for a feature in scene source imagery; systematically rendering said scene containing said high-resolution model; capturing a pixel representation of said rendered scene containing said high-resolution model; and storing said pixel representation of said rendered scene containing said high-resolution model in a database for real time rendering of said scene. In another aspect of the invention a method is presented further comprising the steps of removing polygons having a particular feature code from said polygonal representation of scene to create a reduced polygon representation; and storing said reduced polygonal representation of said scene in a data base for rendering in real time. In another aspect of the present invention a method is presented comprising the steps of: removing coplanar polygons from said polygonal representation of scene to create a reduced polygon representation; and storing said reduced polygonal representation of said scene in a data base for rendering in real time. In yet another aspect of the present invention a method is presented further comprising systematically rendering an orthographic projection of said scene containing high resolution models; removing coplanar polygons from said polygonal representation of scene to create a reduced polygon representation; removing polygons having a particular feature code from said polygonal representation of scene to create a reduced polygon representation and storing said reduced polygonal representation of said scene in a data base for rendering in real time.
In another aspect of the present invention an apparatus is presented for building a database for real time rendering of a scene comprising means for substituting a high-resolution polygonal feature model for a feature in scene source imagery; means for systematically rendering said scene containing said high-resolution model; means for capturing a pixel representation of said rendered scene containing said high-resolution model: and means for storing said pixel representation of said rendered scene containing said high-resolution model in a database for real time rendering of said scene. In yet another aspect of the present invention an apparatus is presented further comprising means for systematically rendering an orthographic projection of said scene containing high resolution models. In another aspect of the present invention an apparatus is presented for building a database for real time rendering of a scene comprising a polygonal representation having coplanar polygons and feature identification codes for polygons comprising: means for substituting a high-resolution polygonal feature model for a feature in scene source imagery; means for systematically rendering said scene containing said high-resolution model; means for capturing a pixel representation of said rendered scene containing said high-resolution model; and means for storing said pixel representation of said rendered scene containing said high-resolution model in a database for real time rendering of said scene. In another aspect of the present an apparatus is presented further comprising means for removing polygons having a particular feature code from said polygonal representation of scene to create a reduced polygon representation, and means for storing said reduced polygonal representation of said scene in a data base for rendering in real time. In another aspect of the present an apparatus is presented further comprising means for removing coplanar polygons from said polygonal representation of scene to create a reduced polygon representation; and means for storing said reduced polygonal representation of said scene in a data base for rendering in real time. In another aspect of the present an apparatus is presented further comprising means for systematically rendering an orthographic projection of said scene containing high resolution models.